Quasi-opportunistic Supercomputing in Grid Environments

The ultimate goal of grid technologies is to materialize the vision of grids as virtual supercomputers of unprecedented power, through utilization of geographically disperse distributively owned resources. Despite the overwhelming success of grids in running pleasantly parallel tasks, there still exists a large set of demanding applications considered the exclusive prerogative of real supercomputers. A few examples include complex systems and weather simulations, computational fluid dynamics and other tightly coupled parallel applications. These rely on a static execution environment with predictable performance, provided through efficient co-allocation of a large number of reliable homogeneously interconnected resources. In this paper, we describe a novel quasiopportunistic supercomputer system that enables execution of demanding parallel applications in grids through identification and implementation of the set of key technologies required to bridge the gap between grids and supercomputers. These technologies include an economic incentive-based framework for establishing and maintaining grid-wise allocation agreements; a co-allocation subsystem that is integrated with the economic framework and enhanced by communication topology-aware allocation mechanisms; a fault tolerant message passing library that hides the failures of the underlying resources; and data pre-staging orchestration.